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(57) ABSTRACT 

An operator input device is configured to provide position 
information based on relative movement of the surface and 
the operator input device. The operator input device includes 
an im^ge detector which is configured to detect an image on 
the surface and provide an image signal corresponding to the 
image detected. A controller is coupled to the image detector 
and is configured to receive the image signal and provide the 
position information based on a position of the image 
relative to the image detector. 
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OPERATOR INPUT DEVICE 
INCORPORATION BY REFERENCE 

The following U.S. Patent is hereby fully incorporated by 
reference: 

U.S. Pat. No. 5,581,094, to Hara et al., entided PHOTO- 
DETECTOR ARRAY COMPRISING 

PHOTODETECTORS, AN OBJECT DETECTOR COM- 
PRISING THE PHOTODETECTOR ARRAY and an 
OBJECT DETECTING PROCEDURE, and assigned to 
Mitsubishi Electric Corporation. 

The present application is based on a provisional appli- 
cation serial. No. 60/073,303 filed on Jan. 28, 1998. 

BACKGROUND OF THE INVENTION 

The present invention relates to an input device for a 
computer system. More specifically, the present invention 
relates to an input device for providing position information 
to the computer system based on movement of the input 
device. 

A traditional computer input device, such as a mouse, 
includes a housing, with a ball mounted in the housing. The 
ball is either configured in a traditional manner in which, in 
the normal work position, the ball engages a work surface 
and rotates based on the user's movement of the mouse 
across the work surface. The ball may also be provided as a 
track ball, which is rotated by digital manipulation from the 
user. In either case, position encoders are used to detect 
rotation of the ball in the mouse, and to provide position 
information indicative of that rotation to the computer. In 
many instances, the position information is used to control 
movement of a visual image (such as a mouse cursor) on the 
display screen of the computer. 

Also, in one prior device, a mouse is configured with the 
track ball arrangement described above. The track ball is 
preprinted with a predetermined image. A charge coupled 
device is used to detect the image on the track ball and detect 
movement of the image. Movement of the predefined image 
is used to provide position information to the computer. 

However, the prior computer mouse which uses the 
charge coupled device configuration has a number of sig- 
nificant disadvantages. First, the reaction time of charge 
coupled devices is quite slow. In addition, processing an 
image signal from a charge coupled device is computation- 
ally intensive and takes a relatively large, and expensive 
processor. Also, charge coupled devices are highly sensitive 
to saturation. In other words, if the ambient light conditions 
are variable, charge coupled devices do not perform well. In 
addition, if an extraneous light source, sucb as a flashlight, 
is directed toward the image producing surface, the charge 
coupled devices can easily become saturated and their 
performance then quickly degrades. 

Further, another prior computer mouse commercially 
available from Mouse Systems of CA included a mouse with 
an LED which was used in conjunction with a mouse pad 
having a predetermined pattern thereon. The pattern was 
formed by a grid of blue and red lines. The emissions from 
the LED was reflected off of the mouse pad to a detector 
which provided an analog output signal. The signal was in 
the form of a waveshape with peaks corresponding to the 
different colored grid lines. From this waveform, the lines 
were counted and interpolated to obtain position informa- 
tion. Such a mouse system requires a special mouse pad with 
a special pattern implemented thereon. 

SUMMARY OF THE INVENTION 
An operator input device is configured to provide position 
information based on relative movement of the operator 
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input device and surface. The operator input device includes 
a pattern detector which is configured to detect a pattern on 
the surface and provide a pattern image signal corresponding 
to the pattern detected. The surface has no predefined pattern 

5 thereon. A controller is coupled to the pattern detector and 
is configured to receive the image signal and provide the 
position information based on a position of the pattern 
relative to the pattern detector. 

In one preferred embodiment, the controller is coupled to 

10 the pattern detector and is configured to receive a primary 
image signal at an initial time and receive a secondary image 
a variable time later. The controller computes and provides 
the position information based on a determination of the 
motion changes observed from the primary image to the 

15 secondary image. 

BRIEF DESCRIPTION OF THE DRAWINGS 

FIG. 1 is a block diagram of an exemplary environment 
for implementing an input device in accordance with the 
present invention. 

FIG. 2 illustrates a computer input device, shown in 
partial sectional and partial block diagram form, in accor- 
dance with one aspect of the present invention. 
25 FIGS. 3A-3F illustrate the movement of images or pat- 
terns detected by the computer input device illustrated in 
FIG. 2. 

FIGS. 4 A and 4B are flow diagrams illustrating the 
operation of a computer input device in accordance with 
30 certain aspects of the present invention. 

FIG. 5 illustrates a multiple source electromagnetic radia- 
tion emitter in accordance with one aspect of the present 
invention. 

FIGS. 6A-6D iUustrate imaging areas for detecting rota- 
tion of the operator input device. 

FIGS. 7A and 7B illustrate an operator input device in 
accordance with another aspect of the present invention. 

FIGS. 8A-^D illustrate detection of images or patterns 
4Q using the operator input device illustrated in FIGS. 7A and 
7B. 

FIG. 9 A is a block diagram of a portion of a user input 
device in accordance with another aspect of the present 
invention. 

45 FIGS. 9B-9E illustrate detection of patterns using the 
user input device shown in FIG. 9 A. 

FIG. lOA is a block diagram of a portion of a user input 
device in accordance with another aspect of the present 
invention. 

FIG. lOB illustrates detection of patterns using the device 
shown in FIG. 10 A. 

FIG. 11 A is a block diagram of a portion of a user input 
device in accordance with another aspect of the present 
invention. 

55 

FIG. UB illustrates detection of patterns using the device 
shown in FIG. IIA. 

FIG. 12 A is a block diagram of a portion of a user input 
device in accordance with another aspect of the present 
go invention. 

FIG. 12B illustrates a waveform generated by a portion of 
the user input device shown in FIG. 12 A. 

DETAILED DESCRIPTION OF THE 
^5 PREFERRED EMBODIMENTS 

The present invention provides a user input device for 
generating position information and providing that informa- 
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tion to a computer system. The position inform alion is including an operating system 35, one or more application 

generated based on detected movement of the user input programs 36, other program modules 37, and program data 

device, or a portion thereof. The movement is detected by 38. A user may enter commands and information into the 

identifying a pattern or image on a surface movable relative personal computer 20 through input devices such as a 

to the user input device and monitoring relative movement 5 keyboard 42 and pointing device 40. Other input devices 

of the pattern. (not shown) may include a microphone, joystick, game pad, 

OVERVIEW satellite dish, scanner, or the like. These and other input 

devices are often connected to the processing unit 21 

FIG. 1 and the related discussion are intended to provide through a serial port interface 46 that is coupled to the 

a brief, general description of a suitable computing envi- ^y^^^^ 23, but may be connected by other interfaces, 

ronment m which the invention may be implemented. guch as a sound card, a parallel port, a game port or/a 

Although not required, the invention will be described, at universal serial bus (USB). A monitor 47 or other type of 

least in part, in the general context of computer-executable jj^pi^y ^^^j^^ ^ connected to the system bus 23 via an 

msu-uctions, such as program modules, being executed by a interface, such as a video adapter 48. In addition to the 

personal computer or other computing device. GeneraUy, ^5 monitor 47, personal computers may typically include other 

program modules mclude routine programs, objects, peripheral output devices such as speaker 45 and printers 

components, data structures, etc, that perform particular ^^q^ shown) 

tasks or implement particular abstract data types. Moreover, ™ , • ^ 1 j 

1 -11 J ■ »u -* -11 • * .u . .u ' personal computer 20 may operate m a networked 

those skilled m the art will appreciate that the invention may • . • 1 • K 

u J -^u *i. . * *• environment using logic connections to one or more remote 

be practiced with other computer system configurations, , . . * >in t-l 

• 1 J* u J u ij J • u' . "^^ computers, such as a remote computer 49. The remote 

including hand-held devices, multiprocessor systems, * In . 1 

* u J ui computer 49 may be another personal computer, a server, a 

microprocessor-based or programmable consumer / . 1 L • .1. * 1 J 

^i«^t™,w « «t„,«,i, or-o .^,;«?^™«,.t<..o r»r,;„f^^„^ router, a network PC, a peer device or other network node, 

electronics, network FCs, minicomputers, mainirame j . • n • 1 j n 1 . j l j 

J *u n T-u • *• -1 1- ui • and typically includes many or all of the elements described 

computers, and the like. The invention is also applicable in u 1 . *u 1 . nu u 1 

j »-u*j . u * 1 above relative to the personal computer 20, although only a 

distributed computing environments where tasks are per- ,c , j • ^-m v -n . . j • t-t>. ^ 

c r • J • * 1- 1 1. '^^ memory storage device 50 has been illustrated m FIG. 1. The 

formed by remote processmg devices that are linked through , . t j • . j • r-i^ ^ • 1 j 1 1 

. . *^ , f , J- * u . J ^- logic connections depicted in FIG. 1 include a local are 

a communications network. In a distnbuted computing . . /t * j -j .1 /^^TAvrx^'* o 1 

, , . , . J- . 1 network (LAN) 51 and a wide area network (WAN) 52. Such 

environment, program modules may be located in both local . i- * . 1 ■ rr- 

, * * J • networking environments arc commonplace m omces, 

and remote memory storage devices. , • • j . ^ 1 • . . j t . 

„„ , „ r.,^ Z t ' r enterprise- wide computer network intra nets and the Internet. 

With reference to FIG. 1, an exemplary environment tor 30 „„ , . , *r 

the invention includes a general purpose computing device ^^hen used m a LAN networking environment, the per- 

in the form of a conventional personal computer 20. includ- computer 20 is connected to the local area network 51 

ing processing unit 21, a system memory 22, and a system ^'^""^^ ^ "^^^^^^^ "^^^.^^^^^ P*^^ ^- ^hen used in a 

bus 23 that couples various system components including WAN networlang environment, the personal computer 20 

the system memory to the processing unit 21. The system 35 1^^^^^^^^^ ^ "^^^^^.^^ ^[ ""^^^^ for estabhsh- 

bus 23 may be any of several types of bus structures mg communicMions over the wide area network 52, such as 

including a memory bus or memory controller, a peripheral ^^^f "^^'^^ ?L^y ^^^^^^^f^ 

bus, and a local bus using any of a variety of bus architec- ^'^^^"^^^^ is connected to the system bus 23 via the serial port 

tures. The system memory includes read only memory f^'^^^ ^ "^^^^^ environment, program modules 

(ROM) 24 a random access memory (RAM) 25. A basic 40 "^^^'""^f V^^"" u ^ ' P'"'^'''"' 

input/output 26 (BIOS), containing the basic routine that "^^^^^ stored in the remote memory storage 

helps to transfer information between elements within the ^T"'^^^' appreciated that the network connections 

personal computer 20, such as during start-up, is stored in exemplary and other means of estabhshmg a 

ROM 24. The personal computer 20 further includes a hard communications hnk between the computers may be used, 

disk drive 27 for reading from and writing to a hard disk (not 45 F^G. 2 illustrates an operator input device, such as mouse 

shown), a magnetic disk drive 28 for reading from or writing Mouse 40 includes housing 102, electromagnetic radia- 

to removable magnetic disk 29, and an optical disk drive 30 'ion source (which may simply be a light source such as an 

for reading from or writing to a removable optical disk 31 L^D) 104, aperture 106 defined in the bottom of housing 

such as a CD ROM or other optical media. The hard disk ^ens 108, image or pattern detector 110, controller 112, 

drive 27, magnetic disk drive 28, and optical disk drive 30 50 ^^^^^ ^^^^^ 40 is shown 

are connected to the system bus 23 by a hard disk drive supported relative to work surface 116. Pattern detector 110 

interface 32, magnetic disk drive interface 33, and an optical ^an be any suitable detector and may be an artificial retina 

drive interface 34, respectively. The drives and the associ- pattem detector. 

ated computer-readable media provide nonvolatile storage Light source 104 can be any suitable source of electno- 

of computer readable instructions, data structures, program 55 magnetic radiation. In one illustrative embodiment, light 

modules and other data for the personal computer 20. source 104 includes LED 118 and lens 120. Radiation 

Although the exemplary environment described herein emitted from an LED 118 is transmitted through lens 120 

employs a hard disk, a removable magnetic disk 29 and a such that it passes through aperture 106 in housing 102 and 

removable optical disk 31, it should be appreciated by those impinges upon work surface 116 which has no predeter- 

skilled in the art that other types of computer readable media 60 opined pattern or image thereon. The light then reflects ofif of 

which can store data that is accessible by a computer, such work surface 116 toward lens 108. Lens 108 collects the 

as magnetic cassettes, flash memory cards, digital video radiation reflected from surface 116 and directs it to image 

disks, Bernoulli cartridges, random access memories detector (e.g., artificial retina) 110. It should be noted that 

(RAMs), read only memory (ROM), and the like, may also lens 108 can be eliminated with the addition of lenses on 

be used in the exemplary operating environment. 65 cither LED 118, image detector 110, or both. 

A number of program modules may be stored on the hard Image detector 110 generates an image signal indicative 

disk, magnetic disk 29, optica I disk 31, ROM 24 or RAM 25, of an image or pattern on work surface 116. The image 
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signal is provided to controller 112 which computes position range (step 149), and adjusting the current source (step 150) 

information based on the image signal. The position infor- are illustrated in the flow diagram of FIG. 4B. 

mation indicates movement of mouse 40 relative to work piGS. 3A-3E and 4A illustrate the operation of mouse 40 

surface 116, as will be described in more detail below. in accordance with one aspect of the present invention. FIG. 
Position information is provided by controller 112, through 5 3A illustrates the 32x32 VSPD (pixel) array on image 

an output such as a cable (not shown), to computer 20 detector 110. onto which the image from surface 116 is 

illustrated in FIG. 1. Mouse 40 may also provide the output directed. The entire viewing area 123 of image detector 110 

to controller 112 through a wireless transmission link such is coincident with the 32x32 pixel array. However, defined 

as infrared ultrasonic, or radiofrequency links. within the entire viewing area 123, is sample area 124. 

In the preferred embodiment, the position information 10 Sample area 124 is smaller than the viewing area and 

provided by controller 112 is provided according to a includes, in one illustrative embodiment, a 10 pixel by 10 

conventional format, such as through a serial interface, a pixel area centered generally around a center 126 of viewing 

universal serial bus (USB) interface, or in any other suitable area 123. 

interface format. In operation, controller 112 first activates source 104 such 

Image detector 110, in one illustrative embodiment, is an that radiation is impinged on work surface 116. This is 

artificial retina manufactured by Mitsubishi Electric Corpo- indicated by block 128 in FIG. 4A. Substantially every 

ration and includes a two-dimensional array of variable surface has some irregularities or surface roughness, if only 

sensitivity photo detectors (VSPDs) which operates in a at the microscopic level. A commercially available artificial 

known manner. Briefly, the VSPDs are formed by a side- retina is capable of resolving images which are only a few 

by-side pair of diodes integrated onto and separated by a microns in size, assuming the light source is powerful 

semi -insulated GaAs layer (pn-np structure). In one enough. Thus, the radiation is reflected off of surface 116 

embodiment, the array is a 32x32 element array, but could back to impinge on viewing area 123, carrying with it 

be larger or smaUer as desired. The photo detector current information indicative of an image or pattern formed by the 

depends, both in sign and magnitude, on applied voltage. surface roughness on surface 116. 

Such VSPDs exhibit an analog memory affect which stores Next, a pattern on work surface 116 is identified, and 

conductivity information when a voltage is applied in the source 104 is optionally deactivated. This is indicated in 

presence of an optical write pulse. This information is blocks 130 and 134. FIG. 3B iUustrates the identification of 

retrieved by injecting an optical readout pulse. an image or pattern 132 within sample area 124 of viewing 

Image processing in such devices is based on optical area 123. As discussed above, identification of the image can 

matrix-vector multiplication. An input image is projected be accomplished using edge extraction or any other suitable 

onto the device as a weight matrix. All VSPDs have one technique. Once image 132 has been identified, data corre- 

electrode connected along rows, yielding a sensitivity con- sponding to that image is stored in memory associated with 

trol vector. Thus, the VSPD sensitivity can be set to arbitrary controUer 112. This is indicated by block 136 in FIG. 4A. 

values in each row within a certain range. In addition, the Next, controller 112 simply waits for a predetermined 

remaining VSPD electrode is connected along columns, time out period. The duration of the time out period deter- 

yielding an output current vector defined by the matrix mines the sampling rate of the system and thus will depend 

vector product of the weight matrix times the sensitivity upon the maximum expected velocity of the mouse. In one 

control vector. illustrative embodiment, the time out duration is approxi- 

In an illustrative embodiment, image detector 110 is 40 naately 1-10 milliseconds. The time out period is indicated 

controlled to perform edge extraction operations. The sen- block 138 in FIG. 4A. 

sitivitiesof two adjacent detector rows arc set to +1 and -1, After the time out period, controUer 112 reactivates 

respectively, whereas all other sensitivities are set at 0. In source 104 (if it has been deactivated), as indicated by block 

this embodiment, the output current is proportional to the 140, and determines whether image 132 has moved. This is 
difference in light intensities of the two active rows. By 45 indicated by block 142. Of course, movement of image 132 

shifting the control voltage pattern in a cyclical manner (0, within viewing area 123 is illustrative of relative movement 

+1, -1,0,0, etc.), the horizontal edges of the input image are between mouse 40 (or at least the image detector 110 of 

sensed. Thus, the system operates in a time sequential and mouse 40) and surface 116 from which the image 132 is 

semi-parallel mode. detected. FIG. 3C illustrates movement of image 132 from 

In one illustrative embodiment, mouse 40 also includes 50 * position, shown in FIG. 3B, to a second position, 

current driver 114 which is coupled to source 104. In that offset from the first position, and illustrated in FIG. 3C, 

embodiment, controller 112 intermittently senses the inten- Based on the movement detected, controller 112 provides 

sity of the radiation generated by source 104 and adjusts the position information in a customary and acceptable format, 

current provided to source 104 through current driver 114. In at an output (such as through a cable). Such position 
other words, if the sensed intensity is lower than a desired 55 information can be used by a computing device, such as that 

range, controller 112 provides a feedback signal to current described with respect to FIG. 1, for any number of things, 

driver 114 to boost the current provided to source 104 in including the movement of a mouse cursor on the monitor 

order to increase the intensity of the electromagnetic radia- 42. Providing the position information is indicated by block 

tion emanating from source 104. If, on the other hand, the 144 in FIG. 4A. 

intensity of the radiation is higher than a desired range, 60 After movement of image or pattern 132 is detected, a 

controller 112 provides the feedback signal to current driver new image or pattern such as image 146, is detected within 

114 to reduce the current provided to source 104 to thereby sample area 124 of viewing area 123. This is indicted by 

reduce the intensity of the radiation emitted from source block 130. Light source 104 may be deactivate and image 

104. This may be done, for example, to reduce the overall data, indicative of the new image 146, is then stored. This is 
power consumption of mouse 40. These steps of activating 65 indicated by blocks 134 and 136. Controller 112 then again 

the radiation source (step 145), determining intensity (step waits for the next time out period and determines whether 

147), determining whether the intensity is within a desired image 146 has moved, such as is illustrated in FIG. 3E. This 
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process continues such that mouse 40 continues to provide representative of a sample image can be written as P(r,6). 
position information indicative of the relative movement of This can be expanded by circular harmonics to: 
mouse 40 and work surface 116. 

+«> Equation 3 

MOVEMENT DETECTION 5 ar,^^)= ^ /f„(r)e'-^ 

Detection of movement of the images in viewing area 123 
can be accomplished in a number of fashions. For example, 

cross correlation can be used to determine movement of the where, 

image within the viewing area 123. Cross correlation i^'^e imaginary unit scalers square root of -1 and each 

between the pattern in the sample area 124 and the pattern parameter of the terms in the summation are calculated as 

for the entire viewing or search area 123 is defined as follows: 
follows: 

1 r2Jr 

^L^tl^ Equation I 

Fix, y) = KY,^nU J)QU -xj- ymj 

-^f -^f First, a circle of radius r is identified in the sample area 

and a Fourier expansion is carried out along the perimeter of 
the circle. Coefficients of the calculated harmonics are 

where: 20 stored. These steps are repeated for that circle in the viewing 

is the length of the sample area 124; area after the time out. The coefficients are then compared to 

(i,j) is position data on the 32x32 pixel array; 0^^^° ^ ^^^^^ indicative of the direction and magnitude of 

P(ij) is the function which indicates light intensity and movement of the image, 

which corresponds to the pattern or image detected; ^^^^ Equation 3 is shown as a sum from 

rxr- '\ ■ c • J- f T u» ■ * -i 25 to +00. However, the sum need only actually be taken 

Q(i,i) IS a function indicative of light intensity over the . *u - r *t. • • 

\it;r^ i-^^. "Oi" zcfo to size of the viewing area. Further, equation 

entire viewing area 123 and ^ . . -n * ^- i. j- . . . 1 t 

fnj^ 3is, m one illustrative embodiment, treated as a Euler 

R(x,y) IS the correlation of P and Q. f^^^^j^^ ^ ^^^^^ calculation simpler, as foUows: 
This Equation can be written in the following summation 

form since the pattern is on the artificial retina cell array <r*=<x)s ewsin e Equation 5 

sam k'areTL ^'""^^^ illustrative embodiment, pattern moment charac- 

^ p' teristics are used to determine movement of the image or 

pattern within the sample area. The pattern in the sample 
^ ^ Equation 2 ^^^^ represented by P(x,y) wherein the sample area 

Fix, >) = 2 Yj -'J^f'"^* ^~y'^^ 35 is a square having sides of length L^. The pattern moment 



p.^ Equation 6 



From Equation 2, it can be determined that in order to m„ = P f\x, y)x''fdxdy 

obtain movement of the image, one must obtain P(ij) first, 40 ~f 
and then after the time out period, obtain Q(i j). Thereafter, 

the values of x and y are moved throughout the entire ^ .... 

viewing area (or pixel region) 123 associated with the image .P ^ ^.^ contmuous integers j_hat can take any 

detector, and Equation 2 is evaluated at each point. The significant value. In general, the 0 order moment, 

maximum of R(x,y) will indicate the point at which the cross 45 ^^^^^ ^ ^ ^ g^^^" ^ 

correlation is maximum, and will thus indicate the distance « rro/ \^ ^ d -t 

, . , , ■ , t T . '"«J J P(^y)d3tdy Equation 7 

which the image has moved. In one illustrative embodiment 

where image detector 110 is implemented as the artificial The O"* order moment is understood to be the total mass 

retina chip from Mitsubishi Electric Corporation, the sum- of the pattern data. The first order moment mjo, mg^, is given 

mation equations set out in Equation 2 can be obtained by so as follows: 

simply summing up the current of one row in the photode- ^ 

tector array at a time. This is equal to one summation of '"lo-JJ Kx,yp^ ^ y 

Equation 2. '«oi=iJP(j'i>')ydxdy Equations 

II should also be noted that, rather than ideotifymg a ^^^^ .^^ ^ ^.^^^^^ ^ ^^^^ 

specific shape, pattern or image in sample area 124, the 55 moment, the coordinates of the center of tnai are obtained 

enture sample area 124 can be used as the image. Then, cross follows* 
correlation is simply determined between that entire image 
and a portion of the array on which that image resides after 
the time out period. 

One potential disadvantage associated with the use of 60 
linear coordinates is that they provide position information 
which varies with rotation of mouse 42. TTierefore, in 
another illustrative embodiment, circular harmonics are used 

to determine movement of the image. In that embodiment. The same process is then carried out using the second 

circular coordinates r and 9 are used instead of linear x,y 65 order moment, m^, rao^ to obtain the principle axis, 

coordinates. The circular coordinates are used to provide In yet another illustrative embodiment, a cord histogram 

position information which is rotationally independent. Data method is used to determine movement. The shape of a 



^ Equation 9 

moo 

moi 

— = yc 

moo 
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pattern can be expressed by a cord histogram such as that wheel 172 also preferably has transparent portions thereof, 
shown in FIG. 3F. Only a single cord 152, and one angle 154 such as notches or slits therein, to allow sonac of the 
are shown in FIG. 3F. However, each cord can be written by radiation to impinge on surface 116. One embodiment of 
its length and angle h(r,6). Assuming each point of pixel data slits 174 is illustrated in FIG. 6B, which is a side view of 
Dpi^ on the shape perimeter is a 1, and the others are 0, then 5 wheel 172. Slits 174 are shown greatly enlarged for the sake 
the requirement of the cord is to satisfy the following of clarity and can take any suitable dimension. Slits 174 are 
equation. illustratively provided in a predetermined pattern which is 

easily recognizable by image detector 110 and controller 
0^x,y)Dpt^(x+n:oz e^+rein e)-l Equation 10 112- However, slits 174 provide a rather large transparent 

area such that radiation can be emitted through the trans- 
The total number of cords is given as follows, since D^^ parent area, onto surface 116, and then be reflected to image 
is defined to take either a 1 or a 0 value: detector 110. In that way, image detector 110 and controller 

112 can obtain signals indicative of images from surface 
/'(r,e>iiDp^x,^)D^(^+rcos e^^in e)cixdy Equation 11 ^^g^ j^j^^j ^^^-^^^ rotational movement of 

To make the computation simpler, assume h is a function 15 ^^^^^ ^'^2. 

of r only. Then one must simply count the number of pixels ^"^^ example, FIG, 8A illustrates a pattern on viewing area 

on the shape perimeter. MathematicaUy speaking, one sim- ^23 caused by slits 174. As in FIGS. 3A-3E, viewing area 

ply sets one point on the shape as the origin, and then draws includes sample area 124 within which patterns are 

circles of radius r around the origin. The number of crossing identified, and from which movemem of the patterns is 

points, of the circles crossing the perimeter, are counted. The 20 determined. However, viewing area 123 also includes a 

radius is increased until there are no crossing points. The predetermmed pattern 176 superimposed thereon because 

crossing points recorded become indicative of the shape of radiation travels only through the transparem portions of 

the perimeter. notches 174. As wheel 172 is rotated, the superimposed 

pattern 176 migrates across viewing area 123. For example, 

DETECTING ROTATIONAL MOVEMENT 25 ^ ^'^^ slightly rotated, then the pattern 176 super- 
imposed on viewing area 123 migrates such as shown in 

nCS. 5 and 6A-6D illustrate another preferred embodi- ^{q 

ment in accordance with the present invention. Since con- Of course, sUts of other shapes could be used as wcU. For 

vemional cotnputer mice are often moved roUitionally as ^ pj^g. 8C and 8D iUustrate that slits 174 can be 

well as Imearly by the user, it may be desirable, m some j rectangular slits. In that case, pattern 178 is super- 

mstances, to have the detection algorithm for detecting ^„ ^^^^ ^^^^ ^^^^^^^ ^^^^^ 

movement of images or patterns be rotationally invariant 173. the pattern migrates across viewing area 123 as iUus- 

such that they do not detect rotational movement of the ^^^^ ^^^^^ ^ ^^^^^^^^ 

mouse. However, m other embodmients it may be desirable controUer 112 to provide position information which not 

to have the algorithm detect rotational movement of the only reflects movement of mouse 40 relative to surface 116, 

mous^. In those latter mstances, it may be desirable to ^^^^ ^ indicative of movement of wheel 172 (or 

modity source 104. ^^^^^^ ^j^^^j ^73) relative to image detector 110 and 

In one illustrative embodiment, where detection of rota- light surface 104. This can all be done without traditional 
tional motion is desired, lens 120 (shown in FIG. 2) is position encoders and associated circuitry and without hard- 
shaped to transmit radiation in an elongate, or elliptical, ^ ware which is used in prior, conventional user input devices, 
pattern 160, as shown in FIG, 6A. Then, if the mouse is piG. 9A is a block diagram illustration of a portion of a 
rotated, pattern 160 moves as shown in FIG, 6B, By making i^p^t device (such as a mouse) 180. Mouse 180 is 
the pattern elongate e.g., eUiptical) rather than simply round, sj^iilar to mouse 170 shown in FIG. 7A, and similar items 
this type of rotational motion can be more easily detected. are similarly numbered. For the sake of clarity, wheel 172 

In addition, FIG. 5 iUustratcs another embodiment of a 45 has been moved from the position shown in FIG. 7A, and a 

light source 162 which can be used in accordance with the niunber of the items shown in FIG. 7A have not been shown 

present invention. Light source 162 actually includes two in FIG. 9A. Mouse 180 not only includes LED 104, but 

point light sources 164 and 166 (which may be LEDs and preferably includes two additional sources of electromag- 

associated lenses) which are used to transmit radiation to the netic radiation, such as LEDs 182 and 184. Further, mouse 

surface being viewed. In that instance, the image pattern 50 180 includes a plurality of light pipes 186 and 188. Each of 

detected by image detector 110 can be used to detect the light pipes 186 and 188 may be formed of a single integral 

most intense points within the image. Those most intense member, or of plurality of members coupled to one another 

points 168 and 170 will correspond to b'ght emitted by a to form the appropriate light path. 

region at or near the center of sources 164 and 166. Thus, In operation, LED 104, lens 108 and image detector 110 

rotation of the mouse will cause rotation of points 168 and 55 operate in a similar fashion to that described above, with 

170 as illustrated in FIG, 6B, Again, this aUows rotational respect to an image located on surface 116. However, in 

motion of mouse 40 to be detected more easUy. accordance with another preferred embodiment, wheel 172 

FIG. 7 A illustrates mouse 170 in accordance with another is provided with a plurality of darkened areas 192 spaced 

illustrative embodiment of the present invention. Mouse 170 about the periphery thereof (as indicated in FIG. 9E). LED 

has components which are similar to those shown in FIG. 2, 60 182 is configured to emit radiation which impinges on an 

and which are similarly numbered. However, mouse 170 area proximate darkened regions 192 on wheel 172. As 

also includes wheel 172. In the illustrative embodiment, wheel 172 is rotated, the alternating darkened regions, and 

wheel 172 is provided on housing 102, such that the operator fight regions 191 which reside between the darker regions 

can rotate wheel 172 to cause additional position informa- 192, reflect the radiation emitted by LED 182. Light pipe 

lion to be provided from mouse 170. In the embodiment 65 188 is configured to receive the radiation reflected by wheel 

shown in FIG. 7A, wheel 172 is interposed in the path of 172 and direct that radiation onto a prespecified portion 194 

electromagnetic radiation emitted by source 104. However, of image detector 110. Since darkened portions 190 are 
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separated by lighter portions 191, as wheel 170 is rotated, ment in which image detector 110 is an artificial retina, the 

the intensity of reflected light, reflected from wheel 172 output is an analog signal which varies over the viewing 

through light pipe 88, will change. Therefore, controller 112 area. FIG, 9C illustrates three waveforms, 208, 210 and 212. 

(shown in FIG. 7A) is configured to examine the prespeci- Waveform 208 is a general depiction of an example of a 
fied area 194 on image detector 110 to determine the 5 waveform provided by image detector 110 which is indica- 

intensity of the light impinging on that area in order to tive of the image on surface 116 within viewing area 123. 

determine whether wheel 172 has rotated, or is being Waveform 208 has a baseline intensity and varies with 

rotated. variation in the image along viewing area 123. The peaks 

Similarly, mouse 180 includes actuator button 196. Actua- and troughs in the signal correspond to different portions of 
tor button 196 is preferably a conventionally arranged press lo the image on viewing area 123. 

button which can be depressed by the user in order to Waveform 210 illustrates the same waveform as wave- 
perform an actuation operation. Thus, actuation button 196 form 208, but with wheel 172 rotated to a position which 
is shown pivotally movable about pivot point 198 in the causes a larger amount of light to be reflected therefrom 
direction indicated by arrow 200. In one preferred through light pipe 188 onto the entire viewing area 123. In 
embodiment, actuation button 196 includes a depending 15 that case, waveform 210 still represents the image detected 
surface 202. LED 184 is configured to direct radiation on siu-face 116, by its shape. However, an offset 214 exists 
emitted therefrom into light pipe 86, which directs that relative to waveform 208. Thus, controller 112 can be 
radiation to a second prespecified portion 204 of image configured to detect offset 214 and interpret that to mean that 
detector 110. wheel 172 has been rotated to a position in which more light 

As actuation button 196 is depressed, surface 202 moves is being reflected therefrom, 

downward to interrupt the emanation of radiation from LED Simflalry, waveform 212 illustrates the same waveform as 

184 to light pipe 186. In one preferred embodiment, surface waveforms 208 and 210, but which also includes an offset 

202 is either opaque, or is formed of a filter material which 216 from waveform 210. Waveform 212 corresponds to a 

filters a portion of the radiation emitted by LED 184. Thus, waveform in which wheel 172 is at a position where it 
when actuator button 196 is depressed, the intensity of light ^5 reflects a larger amount of light, and also where actuator 

emitted on prespecified portion 204 of image detector 110 button 196 is at a position such that it does not impinge upon 

decreases relative to the intensity which impinges on that the radiation emitted from LED 184 to light pipe 186. Since, 

region when actuator button 196 is not depressed. in the embodiment currently being discussed, light pipe 186 

Therefore, controller 112 is preferably arranged to inter- is arranged to emit radiation over the whole viewing area 

mittently examine prespecified portions 194 and 204 to 123, controller 212 can be configured to detect offiset 216 

determine whether wheel 172 is being rotated, and also to and interpret that as meaning that actuation button 196 is in 

determine whether actuation button 196 is depressed. the undepressed position. 

It should be noted that, in some current operator input In one preferred embodiment, offsets 214 and 216 are of 

devices, wheel 172 can also operate as an actuation button, a different magnitude such that controller 112 can distin- 

and can be depressible in much the same way as actuation guish between the two ofkets, and thus determine which 

button 196. Thus, a similar configiu-ation can be provided to event is being indicated by the given offset, 

detect the depression (or actuation) of wheel 172. It should FIG. 9D illustrates yet another technique for detecting the 

also be noted that, rather than providing darkened portions various events which can take place in mouse 180. In the 

190 on wheel 172, those portions could be transparent embodiment illustrated by FIG. 9D, LEDs 104, 182 and 184 

portions. Thus, the amount of light reflected from wheel 190 preferably emit different wavelength radiation. Since an 

would decrease when LED 182 impinges upon one of the artificial retina image detector provides an analog output 

transparent areas. signal, the signal received at the output of the array can be 

¥IG. 9B illustrates a viewing area 123 and sample area filtered to isolate various frequency components of the 
124. FIG. 9B also illustrates the prespecified areas 194 and 45 signal. FIG. 9D illustrates a plot of a signal indicative of 

204 on viewing area 123 which are used to detect rotation of intensity (such as voltage) provided by image detector 110 

wheel 172 and actuation of button 196. Of course, it should plotted against frequency. If the output signal of image 

be noted that any number of prespecified areas can be detector 110 is filtered and subjected to threshold detection, 

provided on viewing area 123, so long as there is also the levels of intensity at various radiation frequencies can be 
sufficient remaining area to detect movement of objects on 50 isolated to provide an indication of which event is then 

surface 116. taking place in mouse 180. For example, FIG. 9D shows 

Other methods could be used to delect movement relative four frequencies, fl, £2, Q, and f4. Assuming that frequen- 

to surface 116, rotation and depression of wheel 172 and cies fl-£3 correspond to the frequencies of radiation emitted 

depression of actuation button 196. For example, if a speci- by LEDs 104, 182 and 184, controller 112 can determine, 

fied section of viewing area 123 is not available, then a 55 upon the intensity of radiation received at those 

histogram algorithm can be used to detect light added by frequency levels, whether wheel 172 is being rotated or 

rotation or depression of wheel 172, and by depression of depressed, and whether actuation button 196 is being 

actuator button 196. Light pipes 186 and 188 are then depressed. 

repositioned to direct the radiation conducted thereby such In another embodiment, the LEDs 104, 182 and 184 can 
that it impinges upon the entire viewing area 123 of image 60 activated in a cyclical fashion. This cycling is synchro- 
detector 110. Thus, the overaU intensity of the radiation nized to the signal provided by image detector 110 and 
impinging on viewing area 123 increases as the light controUer can determine which event is occurring, based on 
reflected from wheel 172, and the light provided by LED the intensity and the sampled time period. 
184, increases. FIG. lOA iUustrates mouse 180 with one modification 
RG. 9C illustrates a graph 206 which plots a signal 65 fi-om that shown in FIG. 9A. Rather than providing separate 
indicative of intensity (such as voltage) against position on discrete LEDs 104, 182 and 184, mouse 180 shown in FIG. 
viewing area 123. FIG. 9C illustrates that, in the embodi- lOA provides only a single LED 104, with a plurality of light 



01/11/2003, EAST Version: 1.03.0002 



us 6,172,354 Bl 

13 14 

pipes 215, 217 and 219 directing a portion of the radiation FIG. 12A is a block diagram of a portion of a user input 

emitted by LED 104 to the specific positions illxistrated. device 250 in accordance with another aspect of the present 

Thus, the separate sources of electromagnetic radiation need invention. A number of items shown in FIG. 12 A are similar 

not be provided. to those shown in previous figures, and are similarly num- 
HG. lOB iUustrates another method of detecting various 5 bered. However, FIG. 12A illustrates that user input device 

events in mouse 180. FIG. lOB shows that prespecified ^'^'^'\°'°f f f 

taA ^nA -^10 J '>iA * J 1 «u 172,detent wheel 256 and detent blocker 258. FIG. 12 A also 

portions 194, 204, 218 and 220 are not arranged along the ^^^^^^^^^^^ ^^^^ ^^^^^^^ ^^^^^^^^ embodiment, 

top of viewing area 123, as illustrated by FIG 9B j^ut are -^^^^^^^ ^^^^^^^ ^^^^^^ ^60, analog-to-digilal (A/D) con- 

insteadarrangedmthecoraersofviewingareal23.Thiscan verier 262 and microprocessor 264. 

be accomplished in one preferred embodiment, by provid- lO ^^^^^^ embodiment, detent wheel 256 is fixedly 

ing hght pipes 186 and 188 (in FIG. 9A) or light pipes 215 ^^^pi^d to wheel 172 by shaft 254. Detent wheel 256 also 

217. and 219 (in FIG. lOA) and any other desired number of preferably includes a plurality of detents 266 on its outer 

hght pipes, and arrangmg those light pipes such that hght periphery. Thus, as wheel 172 rotates, detent wheel 256 is 

emitted thereby is direaed onto the prespecified areas 194, also driven through rotation by shaft 254. 

204, 218 and 220. 15 j^^^^^^ blocker 258, in one preferred embodiment, is a 

FIG. IIA illustrates another embodiment of an input resilient element which engages detents 266 on detent wheel 

device 224 in accordance with another aspect of the present 256. Therefore, as the user rotates wheel 172. detents 266 

invention. A number of items of input device 224 are similar rotate causing corresponding deflection of detent blocker 

to those shown in FIG. 7A, and are similarly numbered. 258. The resilience of detent blocker 258 provides the user 

However, for the sake of clarity, current driver 114 and with a tactile feedback corresponding to rotation of wheel 

controller 112 have been eliminated from FIG. 11 A. Ele- 172. 

ments are also slightly rearranged in HG. IIA relative to Also, in one preferred embodiment, it may desirable to 

those shown in FIG. 7A, again for the sake of clarity. For allow wheel 172 to act as a free wheel, without tactile detent 

example, LED 104 is still arranged to impinge radiation on feedback. Thus, device 250 is provided with mode switch 

surface 116. However, lens 108 and image detector 110 are 252. User actuation of mode switch 252 causes detent 

arranged such that the radiation reflected from surface 116 is blocker 258 to be retracted out of engagement with detents 

reflected upwardly to engage mirror element 226. Mirror 266 on detent wheel 256. This can be accomplished using 

element 226 redirects the radiation back down onto lens 108 any suitable mechanism or other configuration. In that 

where it is coUected and transmitted to image detector 110. configuration, wheel 172 can be fi-eely rotated by the user, as 

Input device 224 also includes a second LED 228 and a track a free wheel, without any tactile feedback, 

ball 230. Track ball 230 is preferably mounted to housing jt should also be noted that the same free wheel configu- 

102 ma conventional manner such that it is rotatable relative .^tion can be obtained using a clutch mechanism (not 

to housing 102 in all directions. shown). In such a clutch mechanism, a first clutch surface 

LED 228 is arranged to emit radiation which impinges on (or clutch wheel) is be rigidly attached to wheel 172 and has 

the surface of track ball 230. The light reflected from the a detent wheel engaging surface facing detent wheel 256. 

surface of track ball 230 is reflected onto lens 108. Control- Detent wheel 256 is preferably movable toward, and away 

ler 112 (not shown in FIG. IIA) is preferably configured to from, the clutch wheel. In order to engage the clutch, detent 

switch between actuation of LED 104 and LED 228, either wheel 256 is moved toward the clutch wheel such that a 

as desired by the user or in a cyclical fashion mentioned mating surface or detent wheel 256 comes into engagement 

3t)0ve, with the clutch wheel thereby causing detent wheel 256 to 

It can thus be seen that, when LED 104 is actuated, user rotate with wheel 172. To disengage the clutch, of course, 

input device 224 can be used as a conventional mouse and detent wheel 256 is moved out of engagement with the 

moved relative to surface 116. Controller 112 then provides clutch wheel and therefore does not turn with wheel 172, 
position information which is indicative of the movement of 45 allowing wheel 172 to turn as a free wheel, 

mouse 224 relative to surface 116. However, when LED 228 FIG. 12A also iUustrates controller 112 in greater detail, 

is activated, user input device 224 can be used much the The analog output of image detector 110, in one preferred 

same as a conventional track ball input device. The user embodiment, is fed into variable resistor 260. The output of 

simply rotates track baU 230 and images on the surface of variable 260 is, in turn, provided to A/D converter 262, A/D 
track ball 230 are detected by image detector 110. Since the 50 converter 262 provides a digital output which is indicative of 

device image detector is surface independent, the mouse baU the input received from variable resistor (or force sensitive 

does not need any specific patterns to be imprinted on the resistor) 260. Microprocessor 264 receives the digital output 

mouseball surface. Controller 110 thus provides position for further processing. 

information which is indicative of the rotation of track ball FIG. 12B iUustrates a waveform 266 indicative of an input 
230. 55 to A/D converter 262. During normal operation, the analog 

It should also be noted that, in a preferred embodiment, an output signal from image detector 110 varies based upon the 

optional opaque separator element 232 can be placed to intensity of light impinging thereon. If, in the example 

separate the radiation which impinges on lens 108 and image illustrated, wheel 172 is continuously rotated such that 

detector 110. In that embodiment, image detector 110 is darkened regions 190 on wheel 172 are continuously passing 
separated into two image detector areas 234 and 236, as 60 through the radiation emitted by LED 104, the output from 

indicated in FIG. IIB. Thus, both LEDs 104 and 228 can be image detector 110, in one preferred embodiment is a 

actuated at the same time, and controller 112 can provide continuously varying analog signal, such as a sinewave 

combined or separated position information indicative not signal indicated by sine wave 266. The amphtude of the 

only of the movement of user input device 224 relative to signal is reduced after passing through variable resistor 260 
surface 116, but also indicative of the user's movement of 65 and is provided to A/D converter 262. 

track ball 230. This turns user input device 224 into a In the configuration illustrated in FIG. 12A, deflection of 

four-degree of freedom device. detent blocker 268 under the force of detents 266 causes 
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variable resistor 260 to change values. This results in the termined sample area, the controller is configured to deter- 

magnitude of the signal 262 to also vary, in a corresponding mine a second position of the image of the first pattern in the 

manner. In one preferred embodiment, defection of detent image detector viewing area around the predetermined 

blocker 258 under the force of one of detents 266 causes the sample area and to provide the position information based 
magnitude of the signal provided to A/D converter 262. In 5 on movement of the image of the first pattern from the first 

FIG. 12B, this corresponds to a higher magnitude signal 268 position in the predetermined sample area to the second 

reaching A/D converter 262 with each deflection of detent position in the image detector viewing area around the 

blocker 258. Thus, the output of A/D converter 262 includes predetermined sample area. 

a digital value which varies as the magnitude of waveform 3. The operator input device of claim 2 wherein the 

266 varies. Microprocessor 264 can then detect the number controller is configured to detect within the predetermined 

of detents or notches through which wheel 172 is rotated. sample area an image of a second pattern, after movement 

This allows microprocessor 264 to obtain rate information, of the image of the first pattern, and to provide the position 

as well the detent information, from the signal. information based on movement of the image of the second 

It should be noted that the present invention can also be pattern from the predetermined sample area into the image 

used with other user input configurations, such as joysticks. detector viewing area surrounding the image detector view- 

The present invention operates without a predetermined ^'■^2- 

pattern on the surface and without expensive charge coupled 4. The operator input de vice of claim 1 wherein the 

devices. It should also be noted that the overall viewing area housing includes an aperture and wherein the image detector 

is preferably approximately 10 mmxlO mm. Any suitable is arranged to detect the image of the first pattern on a work 

area can be used. However, it has been found that an surface, through the aperture. 

appropriate lens for focusing on a much smaller area is more ^ operator input device of claim 1 and further 

expensive. It should be further noted that the solid state comprising: 

artificial retina image detector 110 can be combined on the a movable element mounted to the housing and wherein 

same die as controller 112. the surface is disposed on the movable element. 

Thus, the present invention provides a user input device ^5 ^- operator input device of claim 5 wherein the 

which is capable of delecting movement of the user input movable element comprises: 

device relative to one or more surfaces (when the surface is a track ball rotatably mounted to the housing, 

a work surface such as surface 116, or a surface of a movable 7. The operator input device of claim 5 wherein the 

apparatus mounted on the user input device such as a wheel movable element includes a rotatable wheel mounted for 

or a track ball or an actuator button). The surface need have rotation relative to the housing. 

no predetermined pattern thereon. This is done through 8. The operator input device of claim 5 wherein the 

detection of images or patterns, and monitoring of move- movable element comprises an actuator button movably 

ment of the detected images or patterns on the surface mounted to the housing. 

relative to the user input device. Thus, this is done without 9, The operator input device of claim 5 wherein the 

conventional position encoders and without expensive housing includes an aperture and wherein the image detector 

charge coupled devices. Further, the present invention pro- is configured to detect the image of the first pattern on a 

vides a number of different algorithms for detecting move- work surface, through the aperture, and to detect an image 

ment of the identified pattern, provides feedback to adjust of a second pattern on the surface of the movable element, 

the intensity of the source of electromagnetic radiation, and 10. The operator input device of claim 1 wherein the 

also provides power saving techniques to reduce power pattern detector comprises: 

consumed by the user input device. a source of electromagnetic radiation. 

Although the present invention has been described with H. The operator input device of claim 10 wherein the 

reference to preferred embodiments, workers skiUed in the pattern detector comprises: 

art will recognize that changes may be made in form and an image sensor array arranged to receive radiation car- 
detail without departing from the spirit and scope of the rying pattern information about the surface, 
invention. 12. The operator input device of claim 11 wherein the 
What is claimed is: pattern detector comprises: 

1. An operator input device configured to provide position an artificial retina arranged to receive radiation carrying 
information based on relative movement of a surface and the pattern information about the surface. 

operator input device, the position information being pro- jq 13. The operator input device of claim 10 and further 

vided to control movement of a visual image on a visual comprising: 

display screen of a computing device, the operator input a driver, coupled to the source of electromagnetic 

device comprising: radiation, providing a driving signal to drive the source 

a housing; of electromagnetic radiation; and 
a pattern independent image detector coupled to the 55 wherein the controller is coupled to the driver and is 

housing and having a predetermined sample area posi- configured to detect intensity of the source of electro- 

tioned within an image detector viewing area, the magnetic radiation and provide a feedback signal to the 

image detector being configured to detect within the driver based on the intensity detected to adjust the 

predetermined sample area an image of a first pattern intensity. 

on the surface and provide a pattern signal correspond- 60 14. The operator input device of claim 13 wherein the 

ing to the image of the first pattern detected; and controller is configured to determine whether the intensity 

a controller coupled to the pattern detector and configured detected is within a desired range and adjust the feedback 

to receive the pattern signal and provide the position signal to the driver based on the determination, 

information based on a first position of the image of the 15. The operator input device of claim 10 wherein the 
first pattern within the predetermined sample area. 65 controller is coupled to the source of electromagnetic radia- 

2. The operator input device of claim 1 wherein after tion and is configured to only intermittently actuate the 
detection of the image of the first pattern within the prede- source of electromagnetic radiation. 
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16. The operator input device of claim 10 wherein the 
source of electromagnetic radiation is configured to radiate 
electromagnetic radiation in an elongate pattern, 

17. The operator input device of claim 10 wherein the 
source of electromagnetic radiation includes a plurality of 5 
point light sources. 

18. The operator input device of claim 9 wherein the 
pattern detector further includes: 

a plurality of light pipes directing electromagnetic energy 

from the source to a plurality of surfaces; and 10 
a detection device; and 

a second plurality of light pipes, the second plurality of 
light pipes directing electromagnetic energy reflected 
from the plurality of surfaces to the detection device. 

19. A method of providing information for controlling 
movement of a visual image on a display screen of a 
computing device, the method comprising: 

providing an operator input element movable relative to a 
surface; 20 

detecting an image, in a predetermined sample area of an 
image sensor, of a first pattern on the surface; 

detecting relative movement of the operator input element 
and the pattern on the surface by detecting movement 
of the image of the first pattern into an image detector 25 
viewing area surrounding the predetermined sample 
area; and 

providing position information indicative of the relative 
movement detected. 

20. The method of claim 19 wherein detecting relative •'^ 
movement comprises: 

detecting a first position of the image of the first pattern 
within the predetermined sample area; and 

detecting movement of the image of the first pattern from 
the first position. 

21. The method of claim 20 wherein detecting movement 
of the image of the first pattern from the first position 
comprises: 

detecting a distance of movement of the image of the first 40 
pattern from the first position. 

22. The method of claim 20 wherein detecting movement 
of the image of the first pattern from the first position 
comprises: 

detecting a speed of movement of the image of the first 45 
pattern from the first position. 

23. The method of claim 19 wherein detecting the image 
of the first pattern on the surface comprises: 

impinging electromagnetic radiation on the surface; and 
detecting electromagnetic radiation carrying information 
indicative of the first pattern on the surface. 

24. The method of claim 23 and further comprising: 
extracting the image of the first pattern from the electro- 
magnetic radiation detected. 

25. The method of claim 24 and further comprising: 
intermittently delecting an intensity of the electromag- 
netic radiation; and 

adjusting the intensity detected based on a desired inten- 
sity range. 

26. The method of claim 24 wherein extracting the image 
of the first pattern comprises: 

detecting rotational movement of the first pattern. 

27. The method of claim 26 wherein impinging electro- 
magnetic radiation comprises: 55 

impinging the electromagnetic radiation in an elongate 
pattern. 



28. The method of claim 26 wherein impinging electro- 
magnetic radiation comprises: 

impinging the electromagnetic radiation with a plurality 
of separated radiation sources. 

29. An operator input device configmed to provide posi- 
tion information based on relative movement of a surface 
and the operator input device, the position information being 
provided to control movement of a visual image on a visual 
display screen of a computing device, the operator input 
device comprising: 

a housing including an aperture; 

a movable element mounted to the housing, wherein a first 
surface is disposed on the movable element; 

a pattern independent image detector coupled to the 
housing and being configured to detect a first pattern on 
a work surface through the aperture and to detect a 
second pattern on the first surface disposed on the 
moveable element, the pattern independent image sen- 
sor providing pattern signals corresponding to the first 
and second detected patterns; and 

a controller coupled to the pattern detector and configured 
to receive the pattern signals and to provide the position 
information based upon the pattern signals. 

30. The operator input device of claim 29 wherein the 
pattern detector further includes: 

a plurality of light pipes directing electromagnetic energy 

from the source to a plurality of surfaces; 
a detection device; and 

a second plurality of light pipes, the second plurality of 
light pipes directing electromagnetic energy reflected 
firom the plurality of surfaces to the detection device. 

31. An operator input device configured to provide posi- 
tion information based on relative movement of a surface 
and the operator input device, the position information being 
provided to control movement of a visual image on a visual 
display screen of a computing device, the operator input 
device comprising: 

a housing; 

a pattern independent image detector coupled to the 
housing and being configured to detect a first pattern on 
the surface and provide a pattern signal corresponding 
to the first pattern detected, wherein the pattern detector 
further comprises: 

a source of electromagnetic radiation; and 
a driver, coupled to the source of electromagnetic 
radiation, providing a driving signal to drive the 
source of electromagnetic radiation; and 
a controller coupled to the pattern detector and configured 
to receive the pattern signal and provide the position 
information based on a position of the first pattern 
relative to the pattern detector, wherein the controller is 
also coupled to the driver and is also configured to 
detect intensity of the source of electromagnetic radia- 
tion and provide a feedback signal to the driver based 
on the intensity detected to adjust the intensity. 

32. The operator input device of claim 31, wherein the 
controller is configured to determine whether the intensity 
detected is within a desired range and adjust the feedback 
signal to the driver based on the determination. 

33. An operator input device configured to provide posi- 
tion infonmation based on relative movement of a surface 
and the operator input device, the position information being 
provided to control movement of a visual image on a visual 
display screen of a computing device, the operator input 
device comprising: 
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a housing; 

a rotatable wheel mounted to the housing, wherein the 
surface is disposed on the rotatable wheel; 

a pattern independent image detector coupled to the 
housing and being configured to detect a first pattern on ^ 
the surface and provide a pattern signal corresponding 
to the first pattern detected; and 

a controller coupled to the pattern detector and configured 
to receive the pattern signal and provide the position 
information based on a position of the first pattern 
relative to the pattern detector, wherein the controller is 
configured to detect a second pattern and a position of 
the second pattern relative to the pattern detector, after 
movement of the first pattern, and to provide the 
position information based on movement of the second 
pattern from the position. 

34. An operator input device configured to provide posi- 
tion information based on relative movement of a surface 
and the operator input device, the position information being 
provided to control movement of a visual image on a visual 
display screen of a computing device, the operator input 
device comprising: 

a housing; 

an actuator button movably mounted to the housing, 25 
wherein the surface is disposed on the actuator button; 

a pattern independent image detector coupled to the 
housing and being configured to detect a first pattern on 
the surface and provide a pattern signal corresponding 
to the first pattern detected; and 

a controller coupled to the pattern detector and configured 
to receive the pattern signal and provide the position 
information based on a position of the first pattern 
relative to the pattern detector, wherein the controller is 
configured to detect a second pattern and a position of 
the second pattern relative to the pattern detector, after 
movement of the first pattern, and to provide the 
position information based on movement of the second 
pattern from the position. 

35. An operator input device configured to provide posi- 
tion information based on relative movement of a surface 
and the operator input device, the position information being 
provided to control movement of a visual image on a visual 
display screen of a computing device, the operator input 
device comprising: 

a housing; 

a pattern independent image detector coupled to the 
housing and being configured to detect a first pattern on 
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the surface and provide a pattern signal corresponding 
to the first pattern detected, the pattern detector includ- 
ing a source of electromagnetic radiation configured to 
radiate electromagnetic radiation in an elongate pattern 
onto the surface; and 
a controller coupled to the pattern detector and configured 
to receive the pattern signal and provide the position 
information based on a position of the first pattern 
relative to the pattern detector. 

36. An operator input device configured to provide posi- 
tion information based on relative movement of a surface 
and the operator input device, the position information being 
provided to control movement of a visual image on a visual 
display screen of a computing device, the operator input 
device comprising: 

a housing; 

a pattern independent image detector coupled to the 
housing and being configured to detect a first pattern on 
the surface and provide a pattern signal corresponding 
to the first pattern detected, the pattern detector includ- 
ing a plurality of point light sources configured to 
radiate maximum electromagnetic radiation to spaced 
apart points on the surface; and 

a controller coupled to the pattern detector and configured 
to receive the pattern signal and provide the position 
information based on a position of the first pattern 
relative to the pattern detector. 

37. A method of providing information for controlling 
movement of a visual image on a display screen of a 
computing device, the method comprising: 

providing an operator input element movable relative to a 
surface; 

detecting a pattem on the surface, wherein detecting a 
pattern on the surface comprises: 
impinging electromagnetic radiation on the surface by 
radiating the electromagnetic radiation in an elon- 
gate pattem; 

detecting electromagnetic radiation carrying informa- 
tion indicative of the pattern on the surface; and 

detecting rotational movement of the pattern by extract- 
ing the pattern from the electromagnetic radiation 
detected; 

detecting relative movement of the operator input element 

and the pattern on the surface; and 
providing position information indicative of the relative 

movement detected. 

<f * * « » 
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UNITED STATES PATENT AND TRADEMARK OFFICE 

CERTIFICATE OF CORRECTION 



PATENT NO. : 6, 172,354 Bl Page 1 of 2 

DATED : January 9, 2001 

INVENTOR(S) : Manolito E. Adan et al. 



It is certified that error appears in the above-identified patent and that said Letters Patent is 
hereby corrected as shown below; 



Title page. 

Item [56], OTHER PUBLICATIONS, 

"Genius Optical Mouse", brochure, delete "1988" and insert - 1998 
Column 3 , 

Line 40, after "(ROM) 24", delete "a" and insert -- and 
Column 4 , 

Line 26, after "local", delete "are" and insert -- area --. 
Column 7 , 

Line 62, after "mouse", delete "42" and insert -- 40 --. 

After line 12, equation 1, please replace Equation 1 with the following: 

L I 
2 2 

Column 9 , 

Line 43, before "e.g., elliptical)" insert -- ( 
Column 10 , 

Line 23, before "174", delete "notches" and insert -- slits - . 
Line 35, after "mouse", delete "40" and insert -- 170 --. 

Column 11, 

Line 1, after "wheel", delete "170" and insert -- 172 
Line 3, after "pipe", delete "88" and insert - 188 --. 
Line 17, after "pipe", delete "86" and insert - 186 --. 
Line 41, after "wheel", delete "190" and insert - 172 

Column 12 , 

Line 30, after "controller", delete "212" and insert -- 112 --. 
Line 62, after "controller", insert -- 112 -. 

Column 14 , 

Line 15, after "is", delete "be". 
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UNITED STATES PATENT AND TRADEMARK OFFICE 



CERTIFICATE OF CORRECTION 



PATENT NO. 
DATED 



: 6,172,354 Bl 
: January 9, 2001 



Page 2 of 2 



INVENTOR(S) : Manolito E. Adan et al. 

It is certified that error appears in the above-identified patent and that said Letters Patent is 
hereby corrected as shown below: 



Column 15 , 

Line 2, after "signal", insert - provided from variable resistor 260 to A/D converter — . 
Line 3, delete "defection" and insert -- deflection --. 

Line 4, after "the" second occurrence, insert - resistance of variable resistor 260 to be 
reduced This results in a corresponding increase in the --. 

Line 10, after "detect", insert - not only the direction and rate at which wheel 172 is 
being rotated, but it can also detect --. 

Column 16 , 

Line 17, after "input", delete "de vice" and insert -- device --. 



Signed and Sealed this 



Ninth Day of July, 2002 



Attest: 




JAMES E. ROC AN 



Atiestittg Officer 



Director of the United States Patent and Trademark Office 
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